Cognitive Fit and an Intelligent Agent for a Word Processor: Should Users Take All That Advice?

While intelligent agents have been developed to provide objective and expert advice to users, most experienced users know that they should not be followed blindly. Cognitive fit theory was developed about ten years ago to support the notion that tools should fit the tasks for which they were designed in light of the user's capabilities. Recently, intelligent agents have been provided to nearly every computer user as part of the Microsoft Office Suite. In nearly all of the applications in the suite, suggestions pop up as the software encounters recognized patterns. Users' capabilities vary widely, however. Some users have noticed anomalies in the advice, and their expertise leads them to override that advice. The computer credibility literature would predict that some users will take that advice without questioning it; this paper asserts that this will occur when there is lack of cognitive fit. In this study, the "Advisor," one particular intelligent agent in Microsoft Word was examined. In this experimental study, 33 undergraduate students were exposed to a passage of text with five repetitions each of three types of error conditions: (1) errors flagged correctly, (2) errors found by the Advisor that were not truly errors, and (3) errors missed by the Advisor. Hypotheses were that (1) the Advisor would in general improve performance, (2) Expertise in English would in general improve performance, and (3) the Advisor would help more those with higher English skills than those with lower English skills. Verbal SAT scores were obtained by permission of the subjects to serve as a measure of English skills. Analysis of the data showed that overall, all three hypotheses were supported in general. The paper also provides more detailed results for each of the error types. The results imply the need for careful use of intelligent agents; agents will not substitute for user expertise and could indeed degrade the performance of non-expert users.